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Description 

1 The present invention relates to a method to rate a discrete decoded picture in 
respect to its quality by calculating a picture quality rating function. 

Generally, picture coding/compression standards like JPEG or MPEG are 
5 based on block-based discrete cosine transformation (DCT) encoding. There- 
fore, a loss of quality might arise within an encoded image, such as blocking or 
ringing artefacts. 

Different methods are proposed to reduce such a loss of quality, like from H.W. 

10 Park, Y.L Lee "A Postprocessing Method For Reducing Quantization Effects And 
Low Bit-Rate Moving Picture Coding", IEEE Transactions on Circuit Systems 
For Video Technology, Vol. 9, No. 1, February 1999, pages 161-171, and by S. 
Minami, A. Zakhor "An Optimization Approach for Removing Blocking Effects 
in Tranform Coding", IEEE Transactions on Circuits and Systems for Video 

15 Technology, Vol 5, No. 2, April 1995, pages 74 to 82. 

In particular, according to the first mentioned described postprocessing 
method blocking artefacts are reduced by one-dimensional horizontal and ver- 
tical lowpass filtering and the ringing noise is reduced by two-dimensional sig- 

20 nal-adaptive filtering. Since such restoration techniques for removing blocking 
effects are described to result in unnecessary blurring of the image in the lat- 
ter mentioned paper, a new approach is proposed therein which exploits the 
correlation between the intensy values of boundary pixels of two neighbouring 
blocks. This approach is based on the theoretical and empirical observation 

25 that under mild assumptions quantization of the DCT coefficients of two neigh- 
bouring blocks increases the expected value of the mean squared difference of 
slope (MSDS) between the slope across two adjacent blocks, and the average 
between the boundary slopes of each of the two blocks. Since the amount of 
this increase is dependent upon the width of quantization intervals of the 

30 transformed coefficients among all permissible inverse quantized coefficients 
the set of DCT coefficients which reduces the expected value of this MSDS by 
an appropriate amount is most likely to decrease the blocking effect. 

To judge systems for transmission and/or storing of video signals generally a 
subjective quality of the reproduced moving pictures with a set data rate is 



- 3 - 



Sony International (Europe) GmbH 



1 considered. However, since such a subjective judgement requires a big effort 
different objective models, i.e. algorithms which can be implemented in com- 
puterized systems and rate the picture quality without any human influence, 
are developed to judge different post and/or preprocessing methods as well as 

5 encoding and/or decoding methods for enhancing the picture quality. 

F. Fechter "Ein heuristisch optimiertes Modell zur objektiven Beurteilung der 
Qualitat komprimierter Bildfolgen", 18. Jahrestagung der Fernseh- und Kino- 
technischen Gesellschaft, Erfurt, Tagungsband (1998), Seiten 518-527 dis- 
10 closes such a model which objectively rates the quality of compressed pictures 
by comparing the picture to be rated with the original picture and optimizing 
this model with the help of data retrieved by a subjective test. 

Further, M. Trauberg "Qualitatsmessung von MPEG-codierten Bildfolgen - An- 

15 wendung und Verfahren", 18. Jahrestagung der Fernseh- und Kinotechnischen 
Gesellschaft, Erfurt, Tagungsband (1898), Seiten 505-517 discloses a picture 
quality monitoring in MPEG-based transmission systems which only analyzes 
the processed pictures and calculates a rating for a picture degradation with- 
out knowledge of the original picture. Therefore, a statistical blocking indicator 

20 is determined on basis of the knowledge that the differences inbetween neigh- 
bouring pixels are reduced within coded blocks and are statistically enhanced 
with a high probability across block boundaries. Therefore, to calculate the 
statistical blocking indicator which is a measure for degradation of quality due 
to an MPEG-coding the amplitude values of neighbouring pixels are numeri- 

25 cally evaluated. However, the calculated statistical blocking indicator can 
wrongly indicate blocking where actually no blocking is within the picture 
which leads to a wrongly applied de-blocking filter with a result in loss of qual- 
ity, i.e. smearing. Therefore, the main purpose of the statistical blocking indi- 
cator seems to indicate the result of a block-grid-detection, since it identifies 

30 clearly where within the picture block-boundaries might be. 

However, this proposed method to rate the picture quality without using the 
original signal as reference signal might in some cases strongly deviate from 
the subjective quality rating of the digital processed data. Further, it is de- 
35 scribed that in particular only the quality deterioration resulting from the 
MPEG encoding is rated and therewith only a rating of different source pic- 
tures can be performed in view of the corresponding MPEG-coded pictures and 
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1 not a rating of different encoding/decoding schemes as well as pre- and/or 
postprocessing methods, e.g. performed on the same picture. 

Therefore, it is the object underlying the present invention to provide a method 
5 to rate a discrete decoded picture in respect to its quality which does not only 
reflect deteriorations resulting from the MPEG-coding, but also subjective 
quality improvements based upon different pre- and/or postprocessing meth- 
ods as well as different encoding and/or decoding schemes. 

10 This object is solved by the method to rate a discrete decoded picture in re- 
spect to its quality as defined in independent claim 1. Preferred embodiments 
thereof are respectively defined in the dependent claims 2 to 13. 

Two preferred applications of the method according to the present invention 
15 are defined in claims 14 and 15 and a computer program product embodying 
the method according to the present invention is defined in claim 16. 

The method to rate a discrete decoded picture in respect to its quality accord- 
ing to the present invention is characterized by calculating a picture quality 
20 rating function on basis of information about artefacts within the discrete de- 
coded picture and coding information which was used for discrete coding the 
picture. 

Therewith, the present invention is not only applicable to MPEG coded pictures 
25 but to all kinds of discrete encoded and decoded pictures. Further, the combi- 
nation of considering these two objective features ensures that the rating 
method is adaptable more or less identical to a subjective quality rating of 
digital processed data. 

30 Preferably, said information about artefacts is a criterium of discontinuity and 
said coding information is a scaling factor. In this case, the objective measur- 
able criteria within the decoded picture are chosen so that the most important 
unconsciously considered features during subjective quality rating are re- 
flected. 

35 

Further preferably, said scaling factor is retrieved from the discrete decoded 
picture on basis of a number of bits used for discrete coding the picture. Since 
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1 this number of bits is either preset for different coding methods and/or trans- 
mitted together with the picture data and/or directly retrievable therefrom, the 
scaling factor can be determined independently of the MPEG-bitstream and 
therewith the picture quality rating function calculated according to the 
5 present invention does not use the original signal as reference signal. There- 
fore, the present invention enables the calculation of a picture quality rating 
without reference signal and a low or medium calculation complexity in com- 
parison with the methods according to the prior art. Further, an online (on- 
the-fly) implementation is possible. Of course, this invention is also applicabe 
10 to discrete coding methods that use a transformation which is not based on 
DCT (Discrete Cosine Transform) as long as the coding related information 
might be extracted from the received bit-stream, comparable to the extraction 
of the scaling factor from the MPEG-bit-stream in case of MPEG-coding which 
is based on the DCT. 

15 

Alternatively or additionally, the criterium of discontinuity is based on a rating 
of transitions in-between neighboured blocks of the discrete decoded picture. 
Further preferably, in this case the rating of transitions in-between neigh- 
boured blocks is dependent on at least one respective main gradient and one 
20 respective subgradient of a transition in-between neighboured blocks. 

Therewith, according to the present invention, the discontinuity criteria de- 
scribed by S. Minami and A. Zakhor in "An optimization approach for removing 
blocking effects in transform coding" which is referenced above are used. 

25 

Further preferably, transition in-between neighboured blocks are rated based 
on a sum of a squared difference of a respective main gradient and an interpo- 
lated value of respective subgradients adjacent to the main gradient of a tran- 
sition in-between neighboured blocks. 

30 

Therewith, it is possible to very reliable provide an objective criterium for de- 
termining discontinuity within a picture similar to subjective observations. In 
particular, this method considers a visual effect of block boundaries only in 
case the change at the boundary is bigger than the change within the block. 
35 Therefore, a high gradient of a block boundary is not regarded to be a coding 
artefact in case it is desired, i.e. in case the subgradients also show similar 
values. 
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1 Still further preferably, the rating of transitions in-between neighboured 
blocks is based on a sum of all transitions in-between neighboured blocks. In 
this case, a criterium of discontinuity for the whole picture is built. 

5 The picture quality rating function according to the present invention is pref- 
erably determined distinct in respect to horizontal and vertical transitions. 

Further preferably, said picture quality rating function indicates a maximum 
quality in case the scaling factor indicates a high correlation with the picture 
10 and/or in case the criterium of discontinuity indicates a small discontinuity. 

Still further preferably, said picture quality rating function represents a sum 
of a first function dependent on the criterium of discontinuity and a second 
function dependent on the scaling factor. Therewith, both functions can be de- 
15 termined independently to rate the picture quality. This leads to a particular 
advantage in case the rating is performed on the transmittor side, i.e. encoder 
side of the transmission chain, since in this case the second function which is 
dependent on the scaling factor can be easily retrieved from the encoder of the 
picture and has not to be determined from the decoded picture. 

20 

Further preferably, said first and second functions have the general structure 
fix) = k.e" x /^+d, 

25 with k and y being scaling factors and d represents an offset. 
In this case, further preferably, said first function is defined by 
fl(MSDS)- I00.e- MSDS / 1000 

30 

and said second function is defined by 

/2(M Qu ant) = 100- e - M Quant/5. 

35 In this case, the same tendency of the picture quality rating function is 
achieved as with subjective tests. In other words, the calculated picture qual- 
ity rating function leads to a clear statement which of the tested methods is 
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1 better or not so good in comparison to the other methods. 

The picture quality rating function calculated according to the present inven- 
tion is preferably applicable to discrete cosing transform functions, but - as 
5 stated above - is not limited thereto. 

The method according to the present invention is preferably used to determine 
a preferred picture decoding and/or postprocessing method or accuracy. 

10 Alternatively or additionally, the method according to the present invention is 
preferably used to determine a preferred discrete picture encoding and/or 
preprocessing method or accuracy. 

A computer program product according to the present invention comprises 
15 computer program means adapted to perform all the steps as defined in the 
method to rate a discrete decoded picture in respect to its quality according to 
the present invention or anyone of its preferred embodiments as described 
above when said program is executed on a computer. 

20 Further features and advantages of the present invention are elucidated by the 
following description of preferred embodiments thereof taken in conjuntion 
with the accompanying drawings, wherein 

Figure 1 shows a preferred application of the present invention wherein the 
25 method to rate a discrete decoded picture is used as postprocessing 

control, 

Figure 2 shows the postprocessing control of Figure 1 in more detail with a 
first embodiment of a picture quality rating function according to 
the present invention; 
30 Figure 3 shows a second embodiment of a picture quality rating function ac- 
cording to the present invention; 

Figure 4 shows a third embodiment of a picture quality rating function ac- 
cording to the present invention; 

Figure 5 elucidates the determination of a first input parameter for the pic- 
35 ture quality rating function according to the third preferred em- 

bodiment of the present invention; 

Figure 6 shows the relevant pixels for the calculation of the first input pa- 
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rameter; 

Figure 7 shows a graph representing a number of bits used for coding a pic- 
ture in dependency upon a used scaling factor; 

Figure 8 shows a graph representing the picture quality in dependency on 
the used scaling factor which represents said first input parameter 
of the picture quality rating function according to the third pre- 
ferred embodiment of the present invention; 

Figure 9 shows a graph representing the picture quality in dependency on a 
second input parameter of the picture quality rating function ac- 
cording to the third preferred embodiment of the present invention; 

Figure 10 shows a three-dimensional graph representing the picture quality 
rating function according to the third preferred embodiment of the 
present invention, i.e. the picture quality in dependency on the 
first and second input parameters; and 

Figure 11 shows results of a picture quality rating of not postprocessed and 
postprocessed pictures. 

Figure 1 shows a first preferred application of the method according to the 
present invention, namely the use of the quality rating according to the 
present invention to determine a suitable postprocessing method, e.g. a suit- 
able method for coding artefacts reduction for images/sequences. Basically, 
the picture quality rating is performed within a postprocessing control module 
1 which outputs a control parameter CP which is input into a filter 2 which 
also receives an incoming picture Pin to determine a suitable method of coding 
artefacts reduction so that a picture Pout with enhanced image quality will be 
output from the filter 2. The postprocessing control module 1 also receives the 
incoming picture Pin and coding related information CRI to perform the picture 
quality rating according to the present invention according to which the con- 
trol parameter CP is derived to determine the suitable method of coding arte- 
facts reduction. The determination of a suitable method could e.g. be to apply 
strong or weak filters and/or to select appropriate filters for each (horizontal 
and vertical) block boundary or selected groups thereof within the picture 
seperately. Optionally, the postprocessing control module comprises a further 
input for user control so that the control parameter CP might be determined 
based on user specified parameters. 



Alternatively to this embodiment, the postprocessing control module 1 can also 



Sony International (Europe) GmbH 



1 receive several pictures each of which is already processed with a respective 
different method of coding artefacts reduction to relatively judge which of the 
input pictures has the best quality and will be output to a user. 

5 Figure 2 shows the postprocessing control module 1 shown in Figure 1 in more 
detail. The postprocessing control module 1 comprises a first embodiment of a 
picture quality rating function 3a according to the present invention (first pic- 
ture quality rating function 3a in the following), an artefact measurement unit 
4 and - as mentioned above - optionally user-specified parameters 5. The arte- 

10 fact measurement unit 4 receives the incoming picture Pi n to calculate arte- 
fact-related information ARI which serves as one input parameter to the first 
picture quality rating function 3a. The first picture quality rating function 3a 
further receives the coding-related information CRI as second input parameter 
to calculate the control parameter CP according to which a suitable method of 

15 coding artefact reduction is selected within the filter 2. A switch 6 within the 
post-processing control module 1 is used in case the control parameter CP can 
be determined on basis of user-specified parameters to select whether the out- 
put signal of the first picture quality rating function 3a or one of the user- 
specified parameters 5 is output as control parameter CP. One of the user- 

20 specified parameters 5 is selected according to a user control. 

Figure 3 shows a second embodiment of a picture quality rating function 3b 
according to the present invention. In contrast to the first embodiment shown 
in Figure 2 the coding-related information CRI and the artefact related infor- 
25 mation ARI used to calculate the control parameter CP is not only a respective 
single value, but is respectively represented by N+l single values, namely in- 
formation related to coding by co to cjvj and information related to artefacts by 
ao to sljsj. 

30 Of course, the first and second preferred embodiments of the picture quality 
rating function according to the present invention might also be combined or 
information related to coding might comprise N+l parameters and information 
related to artefacts might comprise M+l parameters, wherein N and M are 
positive integers including 0. 

35 

Therewith, according to the present invention, a numerical value, i.e. the con- 
trol parameter CP, is calculated by the picture quality rating function PQRF 
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1 only based on information that can be extracted from the bit stream and/or 
calculated criteria for a specific kind of artefacts. This numerical value repre- 
sents the picture quality and also reflects the same rating that can be achieved 
by subjective analysis of the according picture. The dependencies of the input 
5 data, i.e. the information related to coding and to artefacts, is highly corre- 
lated with the specific kind of artefact for which this rating function should be 
applied. Therefore, different picture quality rating functions for different kinds 
of artefacts can be designed by providing an environment that combines the 
input information to a single numerical value. In general, the picture quality 
10 rating function can be formulated as following: 

PQRF = /(coding related information, artefact related information) 

as also described in connection with Figure 3. 

15 

Figure 4 shows the picture quality rating function according to a third pre- 
ferred embodiment of the present invention, namely a picture quality rating 
function adapted for blocking artefacts. This third picture quality rating func- 
tion 3c according to the third preferred embodiment of the present invention 

20 which can be applied in order to rate different algorithms for reducing/ remov- 
ing blocking artefacts can be designed by accurate information about this par- 
ticular artefacts and by coding specific information and is therefore in the fol- 
lowing named PQRF-B. In this case, the artefact related information is built by 
a modified form of the mean squared difference of slope (MSDS), which mean 

25 squared difference of slope is designed by S. Minami and A. Zakhor as de- 
scribed in the paper "An Optimization Approach For Removing Blocking Effects 
In Transform Coding" which is referenced above. Further, the coding specific 
information for the PQRF-B 3c is built by a scaling factor Mg U ant of quantiza- 
tion which is directly dependent on the bits needed for coding a block. 

30 

For the PQRF-B 3c which is a function of two input values and which is pref- 
erably a sum of two functions each dependent on one input value, i.e. 

PQRF-B = /(MSDS, MQuant) = /l(MSDS) + /2(Mg ua nt) 

35 

in the following first the criterium of discontinuity MSDS is described and 
thereafter the scaling factor MQuant- 
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1 The gradient in-between two pixels which are adjacent to each over a block 
boundary indicates the change of the pixel values and therefore can be used to 
determine if blocking is present or not. However, no low picture quality shall 
be indicated in case the gradient of two adjacent pixels which are located in 

5 different blocks has a relatively high value, but no blocking is present. There- 
fore, as mentioned above, according to the present invention not only the main 
gradient, i.e. the gradient at the block boundary is observed, but also the sub- 
gradients, i.e. the gradient preceding and succeeding the main gradient. 

10 Figure 5a shows a block boundary with regular gradients inbetween the pixel 
values at the border of two neighbouring blocks and Figure 5b shows a block 
boundary with irregular gradients, i.e. with values of the subgradients consid- 
erably smaller than that of the main gradient, i.e. Figure 5b shows a case in 
which blocking inbetween two neighbouring blocks occurs. 

15 

Since a reliable judgement of blocking can not only be based on the main gra- 
dient, as shown above, according to the present invention the mean squared 
difference of slope is preferably calculated by building a squared difference of 
a respective main gradient and an interpolated value of two respective subgra- 

20 dients adjacent to the main gradient as described in "An optimization approach 
for removing blocking effects in transform coding" by S. Minami and A. Zakhor. 
Since further preferably all pixels at the border of two neighbouring blocks are 
considered to judge whether blocking occurs on that transition 32 pixels serve 
as a basis for calculating an MSDS value for a transition inbetween blocks of 

25 8x8 pixels, as it is shown in Figure 6, namely two rows of pixels adjacent to 
the transitions within each block for a vertical transition and two lines of pix- 
els adjacent to the block boundary for a horizontal transition. 

It is possible to provide different weighting factors for the main gradient and 
30 the mean of the subgradients. 

To calculate a mean squared difference of slope for one block either a mean or 
a sum of all transitions inbetween neighbouring blocks is built. Alternatively, 
the building of the criterium of discontinuity MSDS might also be calculated 
35 distinct in respect to the horizontal and vertical directions. In this case, two 
MSDS values are calculated, namely one representing the mean or sum of all 
vertical transitions and one representing the mean or sum of all horizontal 
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1 transitions. According to the present invention preferrably the criterium of dis- 
continuity for one block is represented by only a single block boundary, re- 
spectively for the horizontal and vertical dirtections. Therefore, in this case the 
needed calculation power is reduced in comparison with the methods accord- 

5 ing to the prior art. 

For MPEG coding the scaling factor MQuant is directly dependent on the bits 
needed for coding one block. Further, it is generally known that an increasing 
scaling factor MQ ua nt leads to an increasing number of DCT coefficients which 

10 are quantized to 0 (zero) and a decreasing scaling factor Mg ua nt leads to a de- 
creasing number of DCT coefficients which are quantized to 0 (zero). Since only 
DCT coefficients are transmitted which are not quantized to 0 (zero) generally 
the number of bits transmitted for one block represents on average the used 
scaling factor. Figure 7 shows a function representing the necessary bits for 

15 coding one block in dependency on the use scaling factor Mg uan t- This curve 
can be expressed by an exponential function with negative exponent which in- 
dicates a maximum number of bits for a scaling factor Mg uan t = 1 and a mini- 
mum number of bits for a scaling factor Mg uan t = 112. Of course, these val- 
ues of the scaling factor which are adapted to MPEG coding and decoding rep- 

20 resent only relative values. 

Since the picture quality rating function according to the third preferred em- 
bodiment of the invention, namely the PQRF-B 3c shall only represent a ten- 
dency of the quality of the rated picture the following features have to be ful- 
25 filled based on the above considerations: 

In case the scaling factor Mg U ant is small no blocking effects will occur 
independently from the criterium of discontinuity MSDS. Therefore, the con- 
sidered transition should be rated as to have a high quality. 

In case the criterium of discontinuity MSDS of the considered transition 
30 is relatively low, the quality thereof should be regarded as high independently 
of the used scaling factor Mguant 

The quality of a picture to be rated decreases with decreasing scaling 
factor MQuant or decreasing criterium of discontinuity MSDS respectively ex- 
ponentially, as it is indicated qualitatively in Figure 8 for the scaling factor 
35 MQ uari t and in Figure 9 for the criterium of discontinuity MSDS. 

Since each of the curves shown in Figures 8 and 9 can be approximated by the 



- 13 - 

Sony International (Europe) GmbH 



1 following function: 

fix) = k-e' x/y +d, 

5 with k and y being scaling factors and d representing an offset it can be con- 
cluded that the PQRF-B 3c has the following structure 

PQRF-B = /(MSDS. M Quan t) = (ki-e- MSDS / a +di)+(k 2 - e - M Quant/P + d 2 ). 

10 This three-dimensional function is shown in Figure 10. 

To bring the PQRF-B 3c into a range so that it corresponds to subjective test 
results, the following values are preferably used for the scaling factors ki, k 2 , 
a and (J and the offset values dj and d 2 : 

15 

ki = k 2 = 100 
di = d 2 = 0 
a = 1000 
(3=5. 

20 

Factors which lie within the range of those values result in a broad rating 
which is similar to that of subjective tests on the same material. According to 
the present invention, preferably an interpolated numerical value is calculated 
which represents the quality of all transitions in horizontal and vertical direc- 

25 tions. Therefore, preferably the picture quality rating function PQRF-B 3c of 
the third preferred embodiment according to the present invention is calcu- 
lated distinct in respect to horizontal and vertical transitions, i.e. one picture 
quality rating function represents a mean value of all horizontal transitions 
and one picture quality rating function represents a mean value of all vertical 

30 transitions. 

Figure 1 1 shows an example of a rating of horizontal transitions of a 50 frame 
picture sequence once for a not postprocessed decoded sequence and once for 
the same, but postprocessed version of the decoded sequence. The picture 
35 quality rating function in respect to the horizontal transitions basically shows 
the same characteristics for both sequences, but in all instances the post- 
processed version of the decoded sequence achieves a higher numerical quality 
value, i.e. control parameter CP. 



